Classical and Quantum Info-manifolds

1 Estimation; the Cramer-Rao inequality Let ρ η (x) be a probability density, depending on a parameter η ∈ R. The Fisher information of ρ η is defined to be [8] G := ρ η (x) ∂ log ρ η (x) ∂η 2 dx. (1) We note that this is the variance of the random variable Y = ∂ log ρ η /∂η, which has mean zero. G is associated with the family M = {ρ η } of distributions, rather than any one of them. This concept arises in the theory of estimation as follows. Let X be a random variable whose distribution is believed or hoped to be one of those in M. We estimate the value of η by measuring X independently m times, getting the data x 1 ,. .. , x m. An estimator f is a function of (x 1 ,. .. , x m) that is used for this estimate. So X is a function of m independent copies of X, and so is a random variable. To be useful, the estimator must be independent of η, which we do not (yet) know. We say that an estimator is unbiased if its mean is the desired parameter; it is usual to take f as a function of X and to regard f as samples of f. Then the condition that f is unbiased becomes ρ η .f := ρ η (x)f (x)dx = η. (2) We use the notation ρ.f for the expectation of f in the state ρ. A good estimator should also have only a small chance of being far from the correct value, which is its mean if it is unbiased. This chance is measured by the variance. Fisher [8] stated, and Rao [22] and Cramer proved, that the variance V of an unbiased estimator f obeys the inequality V ≥ G −1. For the proof, differentiate eq. (2) w. r. t. η to get ∂ρ η (x) ∂η f (x)dx = 1, (3) which can be written as Y (x)(f (x) − η)ρ η (x) dx = ∂ log ρ ∂η (f (x) − η) ρ η (x) dx = 1. (4) We note that this is the correlation of Y and f , so the covariance matrix becomes G 1 1 V. (5) This is positive semi-definite, giving the result. 2 If we do N independent measurements of …